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Intellectual risk is valued among 21 st century skills. Three primary teachers who promoted 
positive learning within mathematical inquiry collaborated with researchers to design and 
apply a rubric to assess children’s progress in taking intellectual risks twice during the year. 

Results suggest that handling setbacks and giving feedback to peers were the most 
challenging skills initially, but showed significant gains by the end of the year. Teacher 
interviews discussed challenges that students faced and how positive classroom culture 
encouraged intellectual risk. 

Emphases on 21st century skills (e.g., www.p21.org) — creativity, critical thinking, 
collaboration and communication — promote inquiry as a way to develop learning and 
innovation skills. Mathematical inquiry is used broadly to describe a process of solving 
complex, open-ended problems. In mathematics, creativity is often neglected, perhaps 
because there is an assumption that there is no place for creativity in mathematics. One 
characteristic of creativity is the capacity to take intellectual risks. Intellectual risk 
involves a willingness to attempt to solve a problem or critique an idea when there is 
uncertainty about the outcome (Atkins, Leder, O’Halloran, Pollard, & Taylor, 1991). To 
promote intellectual risk, a classroom culture is vital as “a social atmosphere in which 
children feel secure enough to play with ideas and to take risks” (Binkley et ah, 2012, p. 
38). Students’ perceptions of support significantly contribute to their willingness to engage 
in intellectual risk (Beghetto, 2009). Beghetto (2009) defines intellectual risk taking in 
science as 

engaging in adaptive learning behaviors (sharing tentative ideas, asking questions, attempting to do 
and learn new things) that place the learner at risk of making mistakes or appearing less competent 
than others. . . . [It] highlights the fact that learning (like most any action) involves uncertainty and, 
therefore, some degree of risk. (p. 210) 

An important aspect of learning mathematics through inquiry is the willingness to take 
intellectual risks. Some forms of mathematical inquiry involve ambiguity in the statement, 
goals or possible solution pathways of the problem (Makar, 2012). This requires students 
to negotiate meanings, including navigating differences of ideas to work towards a 
solution. This process of offering and negotiating ideas can lead to anxiety, however, if 
students fear criticism (Cctin, Ilhan, & Yilmaz, 2014). 

This paper reports on teachers’ assessment of their students’ willingness to take 
intellectual risk. The teachers collaborated with researchers to develop criteria for taking 
intellectual risk within mathematical inquiry, then drafted a rubric to assess students’ 
progress over a year. The rubric also provided them with a framework for developing 
classroom norms to promoted students to take intellectual risks in mathematical inquiry. 


2016. In White, B., Chinnappan, M. & Trenholm, S. (Eds.). Opening up mathematics education research (Proceedings of the 
39th annual conference of the Mathematics Education Research Group of Australasia), pp. 94-101. Adelaide: MERGA. 
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Methodology 


Participants and Study Context 

The participants of the study were three teachers and their students (aged 9-12, n = 84) 
in a suburban primary school in Queensland. The teachers had been teaching mathematics 
through inquiry for at least six years. Recent efforts to improve scaffolding norms of 
inquiry-based argumentation and developing positive learning identity had sensitised them 
to focus on promoting students’ intellectual risk in the classroom (see Makar, Bakker, & 
Ben-Zvi, 2015 and O’Brien, Makar, Fielding-Wells, & Hillman, 2015 for further 
background on these projects). 

Design and Application of Assessment Rubric 

To track progress, the teachers worked with the research team to generate five criteria 
in the areas of innovation, resilience and feedback in which they could assess their 
students’ progress to take intellectual risk: 

• Innovation: Generating and developing ideas 

• Resilience: Handling setbacks 

• Feedback: Giving and responding to feedback 

For each of the five criteria, the teachers described students they had taught over the 
years that struggled with or excelled at each area and through their anecdotal narratives, 
generated qualities in each area at low, middle and high levels of behaviour. Through 
discussion of the narratives, a 3-level rubric (Levels 1-3, with 3 the highest level) was 
created to capture qualities they could observe in students at each level. Level 0 was added 
to accommodate cases where the “low” level was still developing and not yet met (see 
Appendix for descriptors of Levels 1 and 3). 


Table 1 

Inquiry Units Videotaped 


Year level 

Term 2 Inquiry Unit 

Term 4 Inquiry Unit 

Mrs Bluett, Year 4 

Ms Thomson, Year 5 

Ms Jones, Year 6 

How big is 10000? 

Can you predict the sum of 
the angles of a polygon? 
What is the biggest pyramid 
you can make from an A3 
sheet of paper? 

How far can a Year 4 jump? 

What is the best strategy for 
the game ofPlinko? 

Have athletes become 
faster? 


Aided by videotapes of their classrooms during inquiry lessons, the teachers assessed 
each student’s achievement in each criteria in inquiry units that were videotaped in Terms 
2 and 4 of the previous year (Table 1). An intermediate score was applied if students 
demonstrated standards between levels (e.g., a score of 1.5 if Levels 1 and 2 were given for 
different aspects of a criterion). Teachers’ judgement — in particular cases that either were 
challenging or stood out — was discussed in a group interview which was audiotaped 
during a working meeting between the teachers and research team. Each student was 
assigned a score of 0-3 for each criteria from their inquiry lessons in Terms 2 and 4. The 
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data were collated from the three classes to capture students’ levels of intellectual risk, 
both for each criteria and averaged over the five criteria. The purpose of the analysis was 
to assess overall shifts but also to gain insight into elements that appeared easier or harder 
to shift. An overall average score for Intellectual Risk was generated by averaging the 
scores of the five criteria. Hypothesis tests for comparing means were conducted at critical 
level a=0.05. 


Results and Discussion 


Intellectual Risk 2015 box pm 



0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 

Figure 1. Distributions of average scores on Intellectual Risk (scale 0-3) in Term 2 and 4 inquiry lessons 
Figure 1 presents the three combined classes (n = 84 students) distributions of average 
scores for overall levels Intellectual Risk in Terms 2 and 4. The data suggest that overall, 
students showed gains in average levels of Intellectual Risk as measured by the rubric. 

Table 2 shows the results broken down by criteria, with students significantly improving 
by the end of the year with moderate to strong effect sizes (Ferguson, 2009). Table 3 
further summarises the distributions of scores at lower (Level 1 or below) and higher levels 
(above Level 2). While the improvement of these students is encouraging news, a further 
analysis was needed to understand how each criteria may have contributed to this. Each 
criterion with respect to these results are discussed in the subsections below. 


Table 2 

Average (sd) Scores (Scale 0-3, 3 = High) for Intellectual Risk in Terms 2 and 4 


Criteria 

Term 2 
avg (sd) 

Term 4 
avg (sd) 

Avg difference p-value 
(sd) 

Effect 

size 

Generating ideas 

1.38 (0.47) 

2.17(0.52) 

0.79 (0.49) 

<0.01 

1.6 

Developing ideas 

1.25 (0.52) 

2.10(0.55) 

0.85 (0.55) 

<0.01 

1.5 

Handling setbacks 

1.14(0.55) 

2.14(0.55) 

1.00 (0.54) 

<0.01 

1.8 

Responding to feedback 

1.24(0.53) 

2.05 (0.69) 

0.81 (0.58) 

<0.01 

1.4 

Giving feedback 

1.15 (0.68) 

1.91 (0.70) 

0.76 (0.55) 

<0.01 

1.4 

Overall Intellectual Risk 

1.23 (0.45) 

2.07 (0.53) 

0.84(0.41) 

<0.01 

2.0 

Table 3 






Distribution of Student Performance in 

Term 2 and Term 4 for each Criterion 


Criteria 

Term 2 

Term 4 

Term 2 Term 4 



Level 1 or below 

Above Level 2 


Generating ideas 

55% 

5% 

2% 

32% 


Developing ideas 

68% 

10% 

2% 

25% 


Handling setbacks 

77% 

8% 

2% 

29% 


Responding to feedback 

76% 

21% 

2% 

31% 


Giving feedback 

69% 

20% 

3% 

24% 
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Overall Intellectual Risk 


50% 


6% 


2 % 


42% 


Generating and Developing Ideas 

Generating Ideas is about the willingness to offer contributions with the intent to 
either get the group started or to consider a new direction. At the lowest levels, students 
did not contribute (see Appendix). The reasons could have been lack of confidence, but it 
could also be lack of willingness to share. For example, one teacher described a student in 
her class who was a strong student, but saw his ideas as his own. 

Ms Thomson : By the end of the year, ... he showed his mathematical thinking a lot and would have 
the big conversations. . . . But at the beginning of the year he would just sit there ... He wouldn’t 
share anything. He just hid stuff away from everyone. It took me a long time and a lot of discussion 
to get him to a point where [he realised] ‘it’s ok to share’. [19:15, 5 March 2016 working session] 

Although initial ideas may have lacked depth, typically, students found the criterion 
Generating Ideas the least difficult (Table 2, mean = 1.38), with this skill significantly 
higher on average in Tenn 2 than the combined mean of the other four criteria (1.19, sd 
0.48, p = 0.015); however, only 45% of the students initially demonstrated the criterion of 
Generating Ideas above Level 1 (Table 3). Again at the end of the year, the students still 
found the criterion Generating Ideas the least challenging to achieve high levels, with 95% 
achieving above Level 1 (being willing to contribute ideas) and 32% achieving at or near 
the highest level (Level 3: offering new and substantive contributions; encouraging 
contributions from others). By Term 4, the average level of students (2.17, sd 0.52) was not 
significantly higher than the combined mean of the other four criteria (2.05, sd 0.56). This 
suggests that perhaps lower levels of generating ideas are quite accessible but as 
expectations rise, it is no easier to obtain high levels of this criterion than the others. 

Initiating an idea takes intellectual risk, but it’s also a risk to take an unformed idea 
and develop it into a useable one. At the lowest level of Developing Ideas, students were 
able to take directions to complete a task given by someone else (see Appendix). This 
shows that a student is willing to engage with the process of developing an idea. Building 
on or adapting ideas, even at a mid-level (Level 2), was not typical for students in Tenn 2 
(32% of students). This improved to 90% by Tenn 4, with 25% of students (all in the older 
two grades) observed extending ideas substantially (Level 3); only two students (2%) 
demonstrated Level 3 on this skill in Tenn 2. Building on others’ ideas is a skill that 
students find challenging initially and likely need other argumentation skills such as being 
an active listener and learning to justify thinking before they can build on others’ ideas 
(Makar et ah, 2015). 

Handling Setbacks 

Learning always involves setbacks. Students with low tolerance for setbacks are often 
unable to persist in the face of difficulties. In the Term 2 inquiries, students had the most 
difficulty in the criteria Handling Setbacks (mean = 1.14) with only 23% able to 
demonstrate these skills above Level 1 (low, see Appendix). 

As the teachers described students in their class who were challenged by setbacks 
initially, two categories of behaviour stood out for them: (1) students who were fearful of 
mathematics generally (and setbacks reminded them of their struggle to achieve), or (2) 
children who focused on levels of perfonnance and got frustrated by mistakes. Ms 


97 



Thomson spoke of a student fearful of mathematics who used to tear up if she was asked 
any questions in class about mathematics. But by the middle of the year, things changed. 

Ms Thomson : She said, “This is the new me. I can’t believe what 1 was like at the beginning of the 
year”. . . . She really embraced the “I am allowed to make a mistake, and I’m allowed to, but I have to 
do something about it”. And, um, I think she realised that making a mistake or getting something 
wrong, there is just no issue with it. As long as you move on from it and find a better way of doing 
things - and there are heaps of ways of doing things. . . . And so, that sort of language and being able 
to accept that changed her — in everything. [0:24] 

This excerpt was followed by others in which teachers described students who had 
sulked, threw tantrums, cried or withdrew in the first half of the year in the face of 
setbacks. The skill of responding to setbacks showed the greatest improvement over the 
year, with average level of 2.14 (sd 0.55) by the end of the year. By then only 8% of 
students remained at Level 1 or below (Table 3). 

Responding to and Giving Feedback 

Feedback is another aspect of intellectual risk that is influential as feedback is a key 
vehicle for making improvement. Responding to Feedback at Level 1 was recorded as 
being willing to listen to feedback, a skill in which not all students showed evidence. 76% 
of students were at Level 1 (and 2% at Level 0) at the beginning of the year (mean 1 .24, sd 
0.53). The teachers worked with students during the year to help them to understand that 
feedback is not just being critical, but aims to provide them with ways to improve. The 
teachers discussed students who struggled to receive feedback from their peers initially. 

Ms Thomson : He was always offering, he would offer suggestions and things, but no one was 
allowed to challenge him. You know, no resilience whatsoever. ... When someone else was speaking 
he would just sit there. And didn’t listen to others. 

Mrs Bluett: That’s what I’ve found. I’ve had two kids, Liam and Harry, and they were so, in that first 
inquiry they were going their own way and there was no moving them even when people gave them 
feedback, it was still there. Yet their feedback was really good to others. They actually really thought 
through and listened to what other people were saying and gave good feedback to others. ... At the 
end of the year, their work was a lot better than it had been at the first of the year. [10:13] 

These descriptions were common of students who performed well in more 
conventional mathematics lessons. They were accustomed to being right and didn’t believe 
that other students (who didn’t perform as well as they did) could tell them anything 
useful. 

Giving feedback was also very difficult at the start of the year for most students. It 
required students to both be willing to explain and justify their ideas and be active listeners 
(i.e., listening and making sense of their peers’ reasoning), then make connections to their 
own reasoning and offer suggestions. At the beginning of the year, Giving Feedback had 
one of the lowest means (1.15, sd = 0.68), with only 31% of students above Level 1. By the 
end of the year, students still had the lowest levels in both criteria of Responding to 
Feedback (mean 2.05, sd = 0.69) and Giving Feedback (mean 1.91, sd 0.70), with 79% and 
80% of students, respectively, able to demonstrate these skills above Level 1. Students’ 
skills in giving feedback in Term 4 were assessed as significantly lower than the combined 
mean of the other four criteria. This suggests either that this is an area of particular 
challenge for students or that the teachers’ expectation of achievement was set higher than 
the other skills. That is, it is possible that the standards written in the rubric for achieving a 
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middle (Level 2) or high (Level 3) level of skill in responding to or giving feedback were 
more ambitious than in the other areas of intellectual risk. 

Inquiry Experience in Prior Year 

Seven students in the cohort were in an inquiry class the previous year. Although this is 
a small number for a confident and detailed analysis, there are some potential patterns that 
would be worth following up with a larger cohort. 

At the start of the year, the overall average mean level of Intellectual Risk for students 
who were in an inquiry class the previous year (1.75, sd = 0.65) was not significantly 
higher than those without inquiry (1.18, sd = 0.41). This suggests — indeed highlights — that 
children do not necessarily act on these criteria in isolation from the context of the class. 
That is, a student could be more likely to demonstrate taking intellectual risks in a class 
where their peers also do so than in a class where their peers do not. 

Three criteria in Term 2 did stand out for these children, however. (1) First was their 
handling of setbacks. On average, they achieved a significantly higher level (mean 1.79, sd 
0.70) on Handling Setbacks than their peers who were not in an inquiry class the previous 
year (mean 1.08, sd 0.50, p = 0.04). (2) The second was their response to feedback with the 
students who had been in an inquiry class the previous year having a significantly higher 
mean (1.86, sd 0.69) on Response to Feedback than their peers who had not (1.18, sd 0.48, 
p = 0.04). Finally, the students from an inquiry class the previous year had a significantly 
higher level of Giving Feedback (mean 1.86, sd 0.69) than their non-inquiry peers (mean 
1.08, sd 0.65, p = 0.03). What is interesting about these results is that the three areas where 
the students with inquiry the prior year outperformed their peers were in the three skills 
which were typically most difficult to achieve early in the year. Again, the small cohort 
prevents confidence in these outcomes, but further research on this with a larger cohort 
would be beneficial. 

At the end of the year, none of the five criteria showed significant differences between 
students who had been in an inquiry class the prior year and their peers who had not been 
in an inquiry class. This suggests that these skills can perhaps be developed within a single 
school year. 

Intellectual Risk and Classroom Culture 

In Term 2, only two of the 84 students had achieved a rating at Level 3 in any criteria 
for Intellectual Risk as measured by the rubric (see Appendix). These results suggest firstly 
that all five criteria for intellectual risk were difficult for students to demonstrate early in 
the year. There was significant improvement by students in all criteria by Tenn 4. 
However, the shifts across these criteria were not uniform, nor did they happen without 
significant effort from the teacher. 

In the meeting at the beginning of this year when we compiled these data, the teachers 
were setting up their new classrooms and reflected on the process of now developing a 
classroom culture which would develop students’ capacity for taking intellectual risks. 

Ms Jones'. It comes back to your classroom culture, doesn’t it? ... Setting that classroom culture has 

built up a rapport in our classroom that ... it’s ok to have a go. [If] it’s not quite right, [we] do all 

that “roadblocks” [discussion] straightaway now ... And I think having that culture has just 

alleviated a lot of things. ... [25:51] 

Mrs Bluett'. My problem is that . . . you just do too much talk. 
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Ms Thomson : We’ve all said that, though. But you know what? I’ve, I’ve been thinking about that, 
too. That’s why you’ve got that classroom culture. Because [in the beginning] we say it over and 
over and over and over and over and over and over again. I just kept thinking, why am I saying this, 
trying to get these kids to tell me what active listening is — again ? Like I’ve already, I did it two days 
ago, why am I going through that again ! [32:46] 

Mrs Bluett : ... You have to model the talk. You have to teach and you have to model. Because you 
can’t expect kids to do it like that (snap)! [33:45] 

What their discussion emphasises is that intellectual risk is developed through a 
classroom culture which promotes it. And that culture both takes time to develop and 
requires significant scaffolding by the teacher (see Makar et ah, 2015 for Mrs Bluett’s 
scaffolding to develop her classroom culture over a year). 

Limitations of Results 

There are many limitations to this study. First, the scores are based on our retrospective 
self-reports of students’ level skills at two points in the year. Second, the data are from 
students at one school, our own students. The convenience sample used is therefore not 
able to be generalised to all students or classrooms. Third, the sample size is quite small. 
The intent of the analysis is to seek possible patterns rather than to be confirmatory in 
nature. Fourth, the skills described in the five criteria were part of our larger effort to 
develop both norms of inquiry in our classrooms and also students’ positive learning 
identities. The intent is not to suggest that mathematical inquiry develops these skills. 

Conclusion 

If schools are serious about developing 21st century skills then they must find ways to 
promote students’ capacity to take intellectual risks. This study assessed students’ progress 
in developing intellectual risk from two points in the year. Three key ideas can be learned 
from analysing the change in levels for the criteria associated with intellectual risk. First, 
the consistent shift with moderate to strong effect sizes (Ferguson, 2009) in all criteria 
(Table 2) suggest that development of these five skills over the year are achievable. The 
sample size is not high enough to explore whether and how these five criteria are related, 
but the aim is not to examine the psychometric properties of the rubric. The criteria and 
standards in the rubric were created as a pedagogical resource for teachers to support (and 
record) students’ progress and development. Second, the elements of managing setbacks 
and providing feedback to peers appear to be the most challenging for students initially. 
Third, a classroom culture which creates an environment to develop these skills appears to 
be critical. More work in this area is needed. 
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Appendix: Criteria for Intellectual Risk and Descriptors at Two Levels 


Criteria 

High level (Level 3) 

Low level (Level 1) 

Generating 

Offers new or incomplete ideas that 

Waits for others to contribute or 

ideas 

lead to or integrate major (productive) 
ideas 

Supports and encourages contributions 
from others 

contributes ideas that may not be 
relevant or practical 

Acknowledges the ideas of others 

Developing 

Substantively builds on, extends or 

Contributes to working on tasks 

ideas 

adapts others’ ideas 

Keeps group on track towards a solution 

that develop ideas 

Setbacks 

Embraces setbacks as challenges that 
can be overcome 

Uses setbacks to extend, improve or 
alter thinking 

Promotes a culture within the group of 
positive responses to setbacks 

Is aware of setbacks 

Responding 

Seeks feedback or critique 

Listens to feedback but may not 

to feedback 

Evaluates, responds to and/or uses 
feedback to integrate or improve 
(deepen/expand) ideas 

respond to or act on it 

Giving 

Provides substantive feedback that 

Gives surface feedback that may 

feedback 

engages with the issue and has the 
potential to improve the thinking, 
process, evidence or solution 

Seeks clarification to promote 
understanding 

not progress the idea, process or 
solution 
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